Nearly a Year of Nuggets

Charles F Vardeman II

2023-08-28

Where we started…

Where we’re going…

AI Success Factors: Engineering Trust in Deployments

Dr. Vardeman, this is a lot of material!

This is meant to provide something of a “Roadmap”!

First some much older nuggets from the beginning…

Old School AI and the Web…

A vision of “Ontologies”, “Linked Data”, and “Software Agents”…

Tim Berners-Lee, James Hendler, and Ora Lassila. “The Semantic Web.” Scientific American 284, no. 5 (2001): 34–43. https://lassila.org/publications/2001/SciAm.html

Old School “Cool” AI and the Web (2001)…

The semantic web had a vision of Agents with Shared Understanding through Ontologies, the ability to Use Tools Like the Web and Consume a Web of Linked Data as Distibuted Knowledge Graphs.

Tim Berners-Lee, James Hendler, and Ora Lassila. “The Semantic Web.” Scientific American 284, no. 5 (2001): 34–43. https://lassila.org/publications/2001/SciAm.html

What’s an Ontology?

Tom Gruber. “Ontology.” In Encyclopedia of Database Systems, edited by Ling Liu and M. Tamer Özsu, 1–3. New York, NY: Springer, 2016. https://doi.org/10.1007/978-1-4899-7993-3_1318-2.

Ontology Design Patterns

Eva Blomqvist, Pascal Hitzler, Krzysztof Janowicz, Adila Krisnadhi, Tom Narock, and Monika Solanki. “Considerations Regarding Ontology Design Patterns.” Semantic Web 7, no. 1 (November 10, 2015): 1–7. https://doi.org/10.3233/SW-150202.

Google Search and the “Semantic Web”

Google. “Introducing the Knowledge Graph: Things, Not Strings,” May 16, 2012. https://blog.google/products/search/introducing-knowledge-graph-things-not/.

Google. “A Reintroduction to Our Knowledge Graph and Knowledge Panels,” May 20, 2020. https://blog.google/products/search/about-knowledge-graph-and-knowledge-panels/.

Knowledge Graphs

Hogan, Aidan, Eva Blomqvist, Michael Cochez, Claudia d’Amato, Gerard de Melo, Claudio Gutiérrez, Sabrina Kirrane, et al. Knowledge Graphs. Synthesis Lectures on Data, Semantics, and Knowledge 22. Springer, 2021. https://doi.org/10.2200/S01125ED1V01Y202109DSK022.

Knowledge Graphs

What’s a Knowledge Graph?

Hogan, Aidan, Eva Blomqvist, Michael Cochez, Claudia d’Amato, Gerard de Melo, Claudio Gutiérrez, Sabrina Kirrane, et al. Knowledge Graphs. Synthesis Lectures on Data, Semantics, and Knowledge 22. Springer, 2021. https://doi.org/10.2200/S01125ED1V01Y202109DSK022.

AI in 2023..

The “AI Social Disruption”

James Gary, “AI – The Social Disruption”, AI Magazine 42, no. 1 (April 12, 2021), Cover.

Pretrained Foundation Models…

Zhou, Ce, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, et al. “A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT.” arXiv, May 1, 2023. https://doi.org/10.48550/arXiv.2302.09419.

Evolution of Large Language Models

Yang, Jingfeng, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Bing Yin, and Xia Hu. “Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond.” arXiv, April 27, 2023. https://doi.org/10.48550/arXiv.2304.13712.

How to Build a ChatGPT?

“The state of GPT” – You should watch this!

Andrej Karpathy, “State of GPT” | BRK216HFS, Microsoft Build, 2023. https://www.youtube.com/watch?v=bZQun8Y4L2A.

“The state of GPT”

Andrej Karpathy, “State of GPT” | BRK216HFS, Microsoft Build, 2023. https://www.youtube.com/watch?v=bZQun8Y4L2A.

Text to numbers…

“Tiktokenizer.” Accessed August 27, 2023. https://tiktokenizer.vercel.app/.

Base models general representations through “Pre-Training” (GPT)

Radford, Alec, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. “Improving Language Understanding by Generative Pre-Training,” n.d.

Base models general representations through “Pre-Training” (GPT)

Radford, Alec, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. “Improving Language Understanding by Generative Pre-Training,” n.d.

“Pre-Training Datasets?”

LLama: Open and Efficient Foundation Language Models

Touvron, Hugo, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, et al. “LLaMA: Open and Efficient Foundation Language Models.” arXiv, February 27, 2023. https://doi.org/10.48550/arXiv.2302.13971.

LLama: Open and Efficient Foundation Language Models

Touvron, Hugo, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, et al. “LLaMA: Open and Efficient Foundation Language Models.” arXiv, February 27, 2023. https://doi.org/10.48550/arXiv.2302.13971.

Gao et al. “The Pile”?

Gao, Leo, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, et al. “The Pile: An 800GB Dataset of Diverse Text for Language Modeling.” arXiv, December 31, 2020. https://doi.org/10.48550/arXiv.2101.00027.

Gao et al. “The Pile”?

Gao, Leo, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, et al. “The Pile: An 800GB Dataset of Diverse Text for Language Modeling.” arXiv, December 31, 2020. https://doi.org/10.48550/arXiv.2101.00027.

What could go wrong?

What could go wrong?

“Sarah Silverman Sues OpenAI and Meta Alleging Copyright Infringement | CNN Business.” Accessed August 27, 2023. https://www.cnn.com/2023/07/10/tech/sarah-silverman-openai-meta-lawsuit/index.html.

What could go wrong?

“Would I forbid the teaching (if that is the word) of my stories to computers? Not even if I could. I might as well be King Canute, forbidding the tide to come in. Or a Luddite trying to stop industrial progress by hammering a steam loom to pieces.” – Stephen King

Reisner, Alex. “Revealed: The Authors Whose Pirated Books Are Powering Generative AI.” The Atlantic, August 19, 2023. https://www.theatlantic.com/technology/archive/2023/08/books3-ai-meta-llama-pirated-books/675063/.

King, Stephen. “Stephen King: My Books Were Used to Train AI.” The Atlantic, August 23, 2023. https://www.theatlantic.com/books/archive/2023/08/stephen-king-books-ai-writing/675088/.

What About Trusted AI?

We’re Talking about LLMs!

Shanahan, Murray. “Talking About Large Language Models.” arXiv, February 16, 2023. https://doi.org/10.48550/arXiv.2212.03551.